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Abstract 


State and national standards continue to prod teacher education programs 
towards preparing teacher candidates who are capable of integrating com- 
puter technologies into their teaching methodologies. However, providing 
experiences and resources for this type of teacher training necessarily relies 
on students possessing basics skills in computer use and havingK—12 teach- 
ers who modeled productive technology integration in the classroom. While 
some high schools are managing to provide students with these skills and 
experiences, others are not, leaving teacher education programs to address 
the diverse levels of technological skills in their teacher candidates. This 
study presents a description of the categories of computer skill levels repre- 
sented in a freshman class of teacher candidates and how these candidates 
characterize their high school experiences with computers. 


A s we continue to emphasize the importance of technology in educa- 
tion, teachers and administrators are working to address state and 
national (NETS»S) standards in K-12 classrooms (International 
Society for Technology in Education, 2004). Millions of dollars have 
been spent in equipping these classrooms with computers, software, and 
peripherals such as scanners, digital cameras, and printers. Additionally, 
resources have been allocated for technical support and professional 
development for teachers. To complement this emphasis in the K-12 
environment, teacher education programs have been focusing on ISTE’s 
NETS»T (National Technology Standards for Teachers) to prepare new 
teachers to enter K-12 classrooms ready to integrate digital technologies 
effectively (Kelly, 2002). 

For the population of high school students embarking on a career 
in teaching, it is prudent to investigate how they are experiencing the 
transition from the NETS»S to the NETS»T. In other words, are skills 
and dispositions being nurtured in our nation’s high schools that prepare 
teacher candidates to transition into fulfilling the NETS«T? Have these 
high school students been provided with challenging work in their K-12 
tenure that integrated technologies in their curricula? Have they seen good 
models of teaching with technology? Have they been expected to use 
various technology skills and tools in their own learning? The answers to 
these questions can assist institutions of higher education, as they design 
programs that provide a smooth transition for those moving from the 
role of student into the role of teacher. 

Of course, it is unrealistic to expect that high school students aspir- 
ing to become teachers are receiving identical educational experiences 
across the United States. It is possible, however, that upon investigation, 
some trends may be found that shed light on what knowledge, skills, and 
dispositions incoming freshmen teacher candidates bring from their high 
school experiences regarding technology and education. As these pat- 
terns are discovered, they lay the groundwork for further collaboration 
between high school and college faculties to strengthen their respective 


programs to support student achievement in relation to the NETS»S 
and NETS»T. The following paragraphs describe one college’s attempt 
at assessing beginning teacher candidates’ readiness for the NETS»T, and 
the relationship of this readiness to the teacher candidates’ high school 
experiences with technology. 

Our faculty has been collaboratively working toward systematically 
integrating the NETS»T in all our teacher education programs. Figure 1 
graphically depicts our concerted efforts to infuse technologies through- 
out our four-year degree programs. Although we have made significant 
progress over the past six years, the large numbers of students we serve 
complicates our task. Our college of education serves more than 3,200 
teacher education majors within four programs — early childhood, middle 
childhood, adolescent-young adult, and special education. Our institution 
is the largest producer of teacher-education graduates in Ohio, graduat- 
ing more than seven hundred teachers a year. Faculty members were 
frustrated by varying levels of student computer skills evidenced in their 
coursework; although some students were able to complete course assign- 
ments requiring technology use, others needed a tremendous amount of 
tutorial support to create basic presentations or spreadsheets. In an effort 
to insure students would have basic technology skills at the beginning 
of their college career, we implemented an Assessment of Technology 
Competencies or ATC (http://edhd.bgsu.edu/atc/info). Before sharing 
the details of this particular experience in using performance assessment 
to document teacher candidate technology competencies, a brief review 
of the literature in these areas is presented. 

Performance Assessment and Assessing 
Technology Competencies 

In the United States, as well as other nations, educational institutions 
are being carefully scrutinized. Evidence of student progress is expected 
to be documented and distributed (Reeves, 2002; Whittaker & Young, 
2002). The interest in accountability and continuous improvement has 
affected assessment processes in P-16 education, increasing the use of 
standardized tests as well as performance assessments (Bartlett, 2002; 
Brown, 2000; Kimball & Cone, 2002; Maribeth Gettinger-Gaide, 2001; 
Persichitte & Herring, 2002). Performance assessments are characterized 
by their focus on student products or artifacts that demonstrate certain 
skills or achievements that cannot be easily measured through traditional 
standardized tests. Portfolio assessment — and more specifically, electronic 
portfolio development — have grown out of a need for students to collect 
and organize multiple performance assessment products (Holt et al., 2001; 
Quatroche et ah, 2002). 

BGSU teacher education programs, as a part of meeting accreditation 
standards through the National Council for the Accreditation ofTeacher 
Education (NCATE), have been developing key assessments (performance 
assessments) that provide evidence of teacher candidate competency 
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Figure 1: ATC (basic technology skills) & Advanced Technology Skills 


throughout the core curricula. When the need for basic technology skills 
among teacher candidates was identified, developing a performance as- 
sessment to allow students to demonstrate their skills in this area seemed 
a reasonable solution. This assessment would then become a part of the 
key assessment documents that students would compile in an electronic 
portfolio format to document their professional development. 

Some might argue that a skills-based technology assessment is inap- 
propriate in teacher education and that curriculum integration strategies 
and higher-order thinking should be the foci. However, support for tar- 
geting skills first exists both in the current literature and in the ranks of 
teacher education faculty and administration. Early studies by the Apple 
Classrooms of Tomorrow (ACOT) researchers delineated five stages of 
teacher technology use. The ACOT studies identified these stages as entry, 
adoption, adaptation, appropriation, and invention (Sandholtz et al., 
1997). In order for teachers to become effective technology integrators 
in their classrooms, they must first acquire the skills that would support 
the entry and adoption level work. The North Central Regional Educa- 
tional Laboratory’s Digital-Age Literacy initiative (NCREL, 2004) lists 
“technological literacy” as one of the eight vital literacies that students 
must possess to thrive in the 21 st century. 

Up to this point, many attempts at determining student and teacher 
technology competencies have relied on survey data (Collier et al., 2004). 
These self-reports are difficult to evaluate, and researchers cannot adequately 
document technology skill development through such means. K-l 6 educa- 
tors have been grappling with the complexities of assessing technology skills 
in a real-world manner (Engstrom, 2004; Pearson, 2004). Such attempts 
are expensive and difficult to organize. Both IC 3 (Certiport, 2004) and My- 
Target (MyTarget, 2004) are examples of national and regional attempts to 
address teacher technology competencies. Individual institutions of higher 
education are just beginning to develop procedures for identifying student 
computer skills (Gaide, 2004; Graham, 2004). It is within this context that 
our Assessment of Technology Competencies was bom. 

The Assessment ofTechnology Competencies (ATC) 

The first tier of the NETS»T, “Technology Operations and Concepts,” 
specifically points to the development of technology skills for teachers in 
areas of word processing, spreadsheet, presentation, Internet, and media 
(Kelly, 2002). Furthermore, development and demonstration of these 
skills are to be a part of the teacher candidates General Preparation phase 
of their program. Requiring students to demonstrate basic technology 
skills, and giving them support systems to strengthen their development 


of these skills, insures that they are poised to expand their use and under- 
standing of educational technologies as they move into their Professional 
Preparation phase. Student survey data indicate that students are realizing 
the important role computer technologies will play in their future work 
as teachers, and were grateful for the initial prompting to master the basic 
skills they would need for their professional development. 

Currently, the assessment is a four-page document that details the 
construction of three digital products to be completed by the student in 
a proctored, one-and-a-half hour session in the college’s computer lab. 
The products utilize word processing, spreadsheet, presentation, and 
graphics software applications, and integrate Internet and file manage- 
ment expertise (Banister & Vannatta, 2005). These computer skills have 
been identified by ISTE, as well as our faculty, and are considered to be 
essential for first-year education students. Students receive a score based 
on a 40-point scale on completion of the assessment. If students are 
unable to successfully complete the ATC, they are allowed to retake the 
assessment the following month to earn a passing score. 

Although four different versions of the ATC are used in this process, 
one particular version included a prompt in the word processing section 
asking students to write about their high school experiences and computer 
technology. The exact prompt reads, “In the left hand column, write one 
or two paragraphs describing how you used computers in your high school 
classes.” Directors of the ATC began reading these essays and were struck 
by the varying descriptors shared by students. These teacher candidates’ 
responses included statements such as “In my small-town high school, 
we had few computers.” and “In high school we rarefy used computers.” 
In contrast, others related, “In high school I used computers everyday.” 
One student shared, 

In our high school, teachers used computers a lot 
to show what they wanted to teach. Often teachers 
would use PowerPoint presentations to help with their 
teaching. This helps the visual students learn better. 

The teachers often assigned projects that we had to 
use computers to do. 

I personally used computers a lot in many of my 
projects. I made a PowerPoint presentation that was 
shown at a pre-prom assembly about not drinking 
and driving. This was shown to both the junior and 
senior classes. I also had to use computers to type all 
my papers for classes. We had many assignments with 
searching Web sites at our school also. 
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Obviously, these students had very different experiences and percep- 
tions involving technology integration at their high schools. We wondered 
if these perceptions would correlate with the skills students were bringing 
to their college coursework. 

Mode of Inquiry 

In an effort to explore the relationships between student descriptions of 
their high school technology experiences and their technology skills as 
demonstrated on the ATC, we began reading and categorizing their es- 
say responses. We divided students into two groups, one that we labeled 
“High Perception” and the other “Low Perception,” in terms of their 
computer use in high school. 

Two faculty members independently categorized student essays into the 
High Perception and Low Perception categories. Key phrases in student 
responses labeled High Perception of computer use in high school included 
“a lot,” “everyday,” and “many”; listing specific subjects where computers 
were used and mentioning computer classes that were offered at their schools 
also were indications of High Perception. Words used to place students in 
the Low Perception category included “none,” “very little,” and “never.” In 
the event that a student essay did not provide enough information to make 
a judgment on High or Low Perception, that student was dropped from 
the analysis. This occurred for 9 out of 1 10 students during the first round 
of the assessment, and for 1 0 out of 126 for the retake round of the assess- 
ment. After independendy categorizing all of the students, faculty members 
came together and compared results. We established a 1 00% agreement 
rate for the High/Low Perception categories. Examples of responses and 
the categories in which they were placed are noted in Table 1 . 

Obviously, this categorization is a simplistic way to group student 
responses on this topic, and more themes could be identified. Our goal 
was to look at the student scores on the ATC in comparison with their 
descriptions of their high school computer experiences, however, so this 
two-category system allowed us to do that. 

Our next step was to match student scores on their ATC with their 
essay responses. For this task we divided students into two other catego- 


Table 1: Examples of High and Low Perception Statements 

How did you use computers in your high school? 


High Perception 

Low Perception 

"In high school we were required 
to know how to use computers. All 
of our papers and other important 
documents had to be typed on a 
computer in order to be graded. Not 
only were we required to type all of 
our papers, but the school mandated 
that all of the students take a 
technology class" 

"In high school my computer use 
was minimal. 1 basically on ly used 
computers for three things. The 
first was typing school papers. The 
second was browsing the Internet. 
Finally the third was for my semester 
in yearbook." 

"1 used computers in high school in 
many different ways... 1 mainly used 
computers to do research in high 
school." 

"In high school, 1 barely ever had to 
use computers. When 1 did, it was 
for research. No type of computer 
class was mandatory, and 1 had better 
subjects to learn than computers 
skills..." 

"Computers are a very important 
component of any high school 
experience. Not only are there papers 
to write and research, but also there 
are classes that revolve around 
computers..." 

"In my high school we had three 
computer labs and a computer in 
every classroom. We were allowed 
to use the labs for homework 
assignments and research, nothing 
else. The classroom computers 
students weren’t allowed to use at all, 
which we all thought was strange and 
unusual." 


ries, High Score and Low Score. Because the assessment consisted of four 
sections, totaling 40 points, we agreed that a score between 35-40 would 
be considered a High Score. If students scored below 35, they necessarily 
missed critical technology skills in the assessment, and we felt that the 


Low Perception/High Score 

"The technology used during my high school career, wasn’t very complicated. 
There was. Of course, the old pc computer used. We had to do a PowerPoint 
presentation, and fill out a resume using a pre-set format, but other than that, 
there wasn't much technology used. 

Due to the affluence of my city, it was basically just assumed that a student 
had a computer to use back at home and didn’t need to be taught how to use a 
computer, or surf the web. However, in order to graduate with honors, you were 
reuired to take half a credit of computer competency. 1 took a typewriting class." 
40 points on ATC, retake only 

High Perception/High Score 

"Having computers available for student use in high school is a great asset to 
both students and teacher/administration. 1 would always find time in study- 
halls to go down and type up a project or paper that was due next class. It was 
very convenient to be able to access a computer so easily in my school. Class 
activities were always more interesting and fun if they involved computer work. 

1 actually learned more by doing projects, assignment and activities all on a 
computer. 1 noticed that 1 wasn't the only one. Now that 1 look back at it, 1 can 
appreciate it more for the experience of working with computers before 1 entered 
college." 

35 points on ATC, retake only 

Low Perception/Low Score 

"1 used computers in high school mainly on a computer class level, meaning 
that other than that computer class, 1 didn't use any computers at all. 1 took an 
introduction to computer my freshman year and it was just the basics. That class 
was fun, but it was very elementary like how to turn on a computer and how to 
turn it off, basically. 

My senior year was the fun computer class lhad.lt was merely a class for 
outgoing seniors to get an extra credit before getting the diploma. Which was fun 
because everybody just goofed off everyday and had a ball and the teacher didn’t 
care at all because she was right out of college and still knew what it was like to 
be on the other side of the desk." 

15 points on the ATC, retake only 

High Perception/Low Score 

"When 1 was in high school we were required to know howto use computers. All 
of our papers and other important documents had to be typed on a computer in 
order to be graded. Not only were we required to type all of our papers, but the 
school mandated that all of the students take a technology class. 

These classes taught students howto use all otthe windows programs. The 
administration recommended that students take this class as freshmen. This 
allowed students to use the skills they obtained in the technology calss to 
succeed in their other classes." 

0 points on the ATC, first round 
1 1 points on the ATC, retake 


Figure 2: Example Data from Four Quadrants of Students Identified 
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Table 2: Distribution of students within identified categories 


Category 

First Round of ATC 
(100 students*) 

Retake Round of 
ATC (116 students) 

Low Perception/High Score 
(LPHS) 

10 

10% 

12 

10% 

High Perception /High Score 
(HPHS) 

42 

42% 

27 

23% 

Low Perception /Low Score 
(LPLS) 

16 

16% 

20 

17% 

High Perception /Low Score 
(HPLS) 

32 

32% 

57 

49% 


*only one-fourth of the 500 students had the test version with the prompt during 
the first round 


Low Score category was an accurate descriptor of their performance. 
The rubrics for scoring the assessment had been piloted and validated 
the previous year and we were confident that the scores reflected the 
students’ performance on the technology skills targeted. Once the High 
Performance/Low Performance information was crossed with the High 
Score/Low Score data, we identified four categories of students. Figure 2 
notates these: Low Perception/High Score (LPHS), High Perception/High 
Score (HPHS), Low Perception/Low Score (LPLS), and High Percep- 
tion/Low Score (HPLS). 

Using these categories, we aggregated the results. Table 2 presents data 
from student ATC testing for both the first and second round of the as- 
sessment process. During the first round of testing, only one- fourth of the 
students (N=100) wrote about their high school technology experiences, 
as this prompt was only included in one (out of four) of the versions of the 
instrument. All students who didn’t initially pass the ATC were allowed 
to retake the assessment a month later. All of these students (N= 1 1 6) did 
write about their high school experiences, providing comparative data 
for the analysis presented. 

We were quite surprised by this distribution. First, we had hypothesized 
that students relating poor high school experiences with technology would 
do poorly on the assessment (LPLS), and that students documenting a 
high regard for their high school technology experiences would do well 
on the ATC (HPHS). This was true for 58% of the students during the 
first round of testing, but only 40% of the students during the retake 
round. The drop in HPHS students in the retake round can be explained 
partially by the fact that many of the High Perception students passed the 
first round and did not need to retake. Why were students’ perceptions not 
indicators of their abilities to pass a basic skills technology assessment? 

Although 10% of students with poor high school technology experi- 
ences were able to pass the ATC (LPHS), our most intriguing finding was 
the large number of students who believed that they were well prepared 
in high school for technology use, but failed the ATC. The following 
discussion explores the factors that may have affected these results. The 
High Perception- Low Score (HPLS) phenomenon will be addressed first, 
followed by a conversation on the Low Perception-High Score (LPHS) 
group of students. 

High Perception, Low Score (HPLS) 

Students who described their use of technology in high school as strong, 
yet were unable to pass the basic skills test, often described their com- 
puter strengths in two areas: use of word processing and researching the 
Internet. For example: 

At XXXX High School the students used the comput- 
ers frequently. We would often use them during class 
for writing papers, or simply searching the Web. There 
were many computer labs in the school, and almost 


every period they would be packed with students. If 
a teacher needed a computer lab for their class, all 
they had to do was simply reserve the room, and the 
students were free to use the computers. Technology 
was very important at XXXX High School. 

Some students who we categorized as HPLS described their use of 
computers as skillful because of the amount of time they spend on com- 
puters at home. For example: 

In high school I used my computer for many things. 

I spent many nights on the Internet talking with 
my friends using Instant Messenger. I also used the 
computer to complete all my homework assignments. 

Having a computer helped me greatly. I was able to 
hand in documents that were neat and organized 
compared to my handwriting. 

These statements indicate that students are using technology, but the 
computer uses they describe do not provide them with the necessary skills 
they will need in higher education. Faculty in the College of Education are 
expecting students to at least possess the basic technology skills of word 
processing, spreadsheet, presentation, and graphic software along with 
Internet skills and file management. Possession of these skills provides 
for the building block for students to work with other technologies such 
as digital video, handhelds, and electronic portfolio creation. Fluency in 
basic technology skills also provides the stepping stone for the discussion 
of technology integration within teacher education courses. 

It is clear that students’ definitions of what it means to be computer 
competent and the skills required in the NETS»S are not the same. The 
insight into these statements from high school students shows the need 
for collaboration between higher education and school districts to work 
together to address the NETS»S in the K-12 environment. School 
districts that are addressing the NETS»S should be graduating students 
who have a much more realistic perception of their technology skills in 
regards to computer competencies. 

Low Perception, High Score (LPHS) 

Students who described their use of technology in high school as weak 
or nonexistent, yet where able to pass the ATC test, are unique. How are 
students who have a low perception of their computer use in high school 
able to score high on a performance-based computer competency test in 
the first year of higher education? How did students acquire the skills 
necessary to pass the ATC test? 

Some students categorized as LPHS described their schools as lack- 
ing technology, but they may have taken one class where technology 
was used, such as English, yearbook, or keyboarding. This exposure to 
computers may have provided a basis for students to pass the basic skills 
test. For example: 

At XXXX High School, computers were not an im- 
portant aspect or our education. There were very few 
classrooms that had more than one computer. Most 
classrooms did have one computer for the teachers’ 
use. However, there were a few classes that were 
computer based. For example, senior year English 
classes were all in computer labs because it was a 
writing class. Overall, computers were not part of 
our everyday class time at XXXX High. 

With the anticipation that some students may not have had exposure 
in high school to the basic computer skills of the ATC, we put several 
mechanisms into place to help students work on the skills before taking 
the test. First, students were presented with information about the ATC 
during summer orientation before the fall semester. A Web site had been 
created for students to access information concerning the test (http://edhd. 
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bgsu.edu/atc/info/). A copy of a practice test and a practice test rubric 
was provided on the site. Students were able to download the test and 
practice the required basic computer skills before taking the ATC test. 
If students were having difficultly with the practice test, a link to video 
tutorials designed to cover the skills tested on the ATC was also provided. 
The ATC was open book and open note, so students who did practice 
and work on the skills were able to take notes and use them as a guide 
while taking the test. With these self-help instruments in place, students 
were able to learn the basic skills in a relatively short period of time to 
successfully pass the test. 

Students who did not pass the test the first time were able to retake 
any part of the test that they failed. Before students took the test, they 
answered a short survey that addressed how they where able to acquire the 
skills that they failed previously. This survey data (N=417) was collected 
at the beginning of the testing session (Fall 2004) to elicit information 
regarding test preparation and utilization of ATC information. Although 
participants had received a packet of information describing the ATC 
during summer orientation, to our surprise 38% indicated that they did 
not read it. Only 16% of the participants read the packet and used it 
to prepare for the assessment. Students also received much information 
regarding the ATC during the first couple weeks of class. This informa- 
tion detailed resources available for ATC preparation. On the survey, 
five options of support were listed: practice test, online tutorials, support 
sessions, Student Tech, and other. Of the 417 survey participants, 67% 
used the practice test to prepare, while only 3-5% utilized support ses- 
sions or Student Tech. Interestingly, 32% used the online tutorials, and 
43% used “other” resources, which typically referred to a knowledgeable 
friend or classmate. The top resource used to prepare for the retake was 
“other” (57%). Fifty percent of retake participants continued to utilize 
the practice test as well. The survey data suggests that perhaps students 
who had a low perception of their high school technology use and lacked 
technology proficiency were able to use various tools to acquire the basic 
ATC skills on their own, in a short amount of time. 

Conclusion 

From our experience in assessing teacher candidates in basic technology 
skills, it is clear that their exposure to the NETS»S is quite varied. The 
integration of digital technologies in their numerous high school environ- 
ments ranges from exceptional to non-existent. Although this disparity 
is worrisome for all high school students, it is especially problematic for 
those aspiring to be teachers. The K-12 students who are not seeing 
technology integration being modeled in their schools are less likely to 
value this emphasis once they reach their teacher education programs. 
Colleges of education will have a difficult time convincing these students 
of the importance of investing efforts in demonstrating the NETS»T. If we 
are dedicated to truly transforming classrooms through the effective use 
of technologies in teaching and learning, then high schools and colleges 
need to work together more closely to find ways to bridge the gaps. 

This study did not extend to the point of identifying specific high 
school programs and their uses of computer technologies. Research of this 
type needs to be conducted, connecting student performance in college to 
their previous experiences in high school. Researchers need to go beyond 
the students’ self-reports and begin collecting data from schools concern- 
ing their technology experiences for students. Continuing to compare 
the high school program data with the student college performance data 
could help identify programs that are working. These programs could 
potentially be used as models for other high schools. 

Although this study does show that college students can quickly acquire 
a set of basic computer skills after high school, it is more difficult to con- 
vince these students of the importance of technology integration within 
K-12 classrooms. Future teachers need to see effective K-12 computer 


utilization while they are students in this environment. Teacher education 
faculty with expertise in technology integration strategies should commu- 
nicate with high school guidance counselors regarding the importance of 
technological competencies for prospective teachers. These higher educa- 
tion faculty members could serve as resources for high school teachers 
who desire more support in implementing the NET»S and modeling the 
NETS»T. High school and college faculties could begin to collectively 
strategize to create dynamic and effective technology experiences for 
students that support teaching and learning. Nurturing a dialogue that 
is open and mutually supportive, for the benefit of future teachers, could 
help these teacher candidates build a bridge from the NETS»S to the 
NETS»T. This type of collaboration has the potential to powerfully affect 
K-12 and teacher education. How shall we proceed? 
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